1. Technical Field
The present disclosure generally relates to the field of combining real scene elements from a camera, such as a television type camera, with virtual scene elements from a virtual camera into a finished composite image. More specifically, the present disclosure is directed towards methods for working with the imperfectly lit blue or green screen backgrounds typically found in composite video and motion picture production.
2. Background
The state of the art in combining real world imagery with additional imagery from another source is a process that requires considerable precision. The most common process used to combine the real world or live action imagery with virtual imagery uses a blue or green background against which the live action subject is photographed. The live action image is then processed to remove the blue or green background, and is then combined with a different background image to produce a final composite image. The blue or green removal process is typically called “keying.”
Several types of keying systems are in common use, among them are chroma keying, luma keying, and color difference keying. Since the basic process behind most of the keying methods is comparing some value of the foreground image against the value of the background image, they share a common requirement for a very evenly lit background image in order to generate an accurate separation between the foreground and background image. If the background is unevenly lit, and the separation parameters in the keying control are increased enough to completely remove the background from the live action image, the keying system will typically remove significant sections of the live action subject, typically including highly visible elements, such as fine hair detail. This destroys the illusion of the composite image and is a main contributor to the artificial appearance of poorly made composite images.
This requirement is complicated by the physics of lighting large flat backgrounds. Since a given light source lowers in lighting intensity as a function of the distance from the source, lights tend to produce a bright area on the background area nearest to the light source, and a darker area on either side. A variety of methods have been employed to try to even out the background lighting and thus improve the keying performance, but achieving a perfectly evenly lit background is a notoriously difficult and time consuming task, and contributes greatly to the expense of doing composite motion picture photography.
One method of correcting lighting variations is to use a reference image of the lit background taken before there are live action subjects placed in front of it. This reference image contains the same background lighting variations that are found in the live action foreground image. Since the lighting variations are then known, the live action image's background can be normalized by dividing the live action image by the reference image, and then multiplying the result by the average color of the reference image. This results in a live action image with only some of the background lighting variation removed.
The above technique does not provide good results. Further, for a shot that involves a moving camera, which is very common in the motion picture and television industries, the above technique will not work, as the reference background image is no longer accurate as soon as the live action camera is moved.